CNC machine - Tool Wear Forecasting

1. Discription

CNC Machining data was collected from a CNC machine for variations of tool condition, feed rate, and clamping pressure.

  • feed rate

    • relative velocity of the cutting tool along the workpiece (mm/s)

  • clamping pressure

    • pressure used to hold the workpiece in the vise (bar)

Data was collected with a sampling rate of 100 ms from the 4 motors in the CNC (X, Y, Z axes and spindle).
And the output includes tool condition (unworn and worn tools) and whether or not the tool passed visual inspection.
This dataset used for tool wear detection or detection of inadequate clamping and tool wear forcasting.

Analysis

  • Exploitory Data Aanalysis
  • Data visualization (especially using bokeh via holoviews)
  • feature enginieering technique
    • Lag features
    • Differential feature
  • modeling technique
    • LightGBM
  • Causal analysis skill
  • Frequency analysis skill
    • FFT

2. Import libraries

In [60]:
import numpy as np
import os
import pandas as pd
import seaborn as sns
import holoviews as hv
from holoviews import opts
hv.extension('bokeh')
from matplotlib import pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, LabelEncoder
from sklearn.metrics import accuracy_score, f1_score, roc_auc_score
import os
from scipy import signal
import lightgbm as lgb
import shap
import warnings 
warnings.filterwarnings('ignore')

3. Load the dataset

In [68]:
data_result = pd.read_csv("train.csv")
print('shape',data_result.shape)
data_result.head(3)

shape (18, 7)
Out[68]:
No material feedrate clamp_pressure tool_condition machining_finalized passed_visual_inspection
0 1 wax 6 4.0 unworn yes yes
1 2 wax 20 4.0 unworn yes yes
2 3 wax 6 3.0 unworn yes yes
In [61]:
experiment_tmp = pd.read_csv("data_01.csv")
print(f'data_XX.csv : {experiment_tmp.shape}')
print(experiment_tmp.columns)
experiment_tmp.head(3)

data_XX.csv : (1055, 48)
Index(['X1_ActualPosition', 'X1_ActualVelocity', 'X1_ActualAcceleration',
       'X1_CommandPosition', 'X1_CommandVelocity', 'X1_CommandAcceleration',
       'X1_CurrentFeedback', 'X1_DCBusVoltage', 'X1_OutputCurrent',
       'X1_OutputVoltage', 'X1_OutputPower', 'Y1_ActualPosition',
       'Y1_ActualVelocity', 'Y1_ActualAcceleration', 'Y1_CommandPosition',
       'Y1_CommandVelocity', 'Y1_CommandAcceleration', 'Y1_CurrentFeedback',
       'Y1_DCBusVoltage', 'Y1_OutputCurrent', 'Y1_OutputVoltage',
       'Y1_OutputPower', 'Z1_ActualPosition', 'Z1_ActualVelocity',
       'Z1_ActualAcceleration', 'Z1_CommandPosition', 'Z1_CommandVelocity',
       'Z1_CommandAcceleration', 'Z1_CurrentFeedback', 'Z1_DCBusVoltage',
       'Z1_OutputCurrent', 'Z1_OutputVoltage', 'S1_ActualPosition',
       'S1_ActualVelocity', 'S1_ActualAcceleration', 'S1_CommandPosition',
       'S1_CommandVelocity', 'S1_CommandAcceleration', 'S1_CurrentFeedback',
       'S1_DCBusVoltage', 'S1_OutputCurrent', 'S1_OutputVoltage',
       'S1_OutputPower', 'S1_SystemInertia', 'M1_CURRENT_PROGRAM_NUMBER',
       'M1_sequence_number', 'M1_CURRENT_FEEDRATE', 'Machining_Process'],
      dtype='object')
Out[61]:
X1_ActualPosition X1_ActualVelocity X1_ActualAcceleration X1_CommandPosition X1_CommandVelocity X1_CommandAcceleration X1_CurrentFeedback X1_DCBusVoltage X1_OutputCurrent X1_OutputVoltage ... S1_CurrentFeedback S1_DCBusVoltage S1_OutputCurrent S1_OutputVoltage S1_OutputPower S1_SystemInertia M1_CURRENT_PROGRAM_NUMBER M1_sequence_number M1_CURRENT_FEEDRATE Machining_Process
0 198.0 0.0 0.00 198.0 0.0 0.000000 0.18 0.0207 329.0 2.77 ... 0.524 2.740000e-19 329.0 0.0 6.960000e-07 12.0 1.0 0.0 50.0 Starting
1 198.0 -10.8 -350.00 198.0 -13.6 -358.000000 -10.90 0.1860 328.0 23.30 ... -0.288 2.740000e-19 328.0 0.0 -5.270000e-07 12.0 1.0 4.0 50.0 Prep
2 196.0 -17.8 -6.25 196.0 -17.9 -0.000095 -8.59 0.1400 328.0 30.60 ... 0.524 2.740000e-19 328.0 0.0 9.100000e-07 12.0 1.0 7.0 50.0 Prep

3 rows × 48 columns

4. Pre-processing

NaN in passed_visual_inspection in experiment result means machining_finalized is no, which means machining process was not finished correctly and did not proceed to visual inspection process. So, we need to fill NaN with no.

In [69]:
data_result['passed_visual_inspection'] = data_result['passed_visual_inspection'].fillna('no')

adding each experiment settings and result to experiment time series data to make one total dataframe.

In [72]:
frames = []
for i in range(1,19):
    #load files
    exp_num = '0' + str(i) if i < 10 else str(i)
    frame = pd.read_csv(f"data_{exp_num}.csv")

    #load each result row
    exp_result_row = data_result[data_result['No'] == i]
    frame['exp_num'] = i

    #add settings to features
    frame['material'] = exp_result_row.iloc[0]['material']
    frame['feedrate'] = exp_result_row.iloc[0]['feedrate']
    frame['clamp_pressure'] = exp_result_row.iloc[0]['clamp_pressure']
    
    #add result to features
    frame['tool_condition'] = exp_result_row.iloc[0]['tool_condition']
    frame['machining_finalized'] = exp_result_row.iloc[0]['machining_finalized']
    frame['passed_visual_inspection'] = exp_result_row.iloc[0]['passed_visual_inspection']

    frames.append(frame)

df = pd.concat(frames, ignore_index = True)
df.head(3)
Out[72]:
X1_ActualPosition X1_ActualVelocity X1_ActualAcceleration X1_CommandPosition X1_CommandVelocity X1_CommandAcceleration X1_CurrentFeedback X1_DCBusVoltage X1_OutputCurrent X1_OutputVoltage ... M1_sequence_number M1_CURRENT_FEEDRATE Machining_Process exp_num material feedrate clamp_pressure tool_condition machining_finalized passed_visual_inspection
0 198.0 0.0 0.00 198.0 0.0 0.000000 0.18 0.0207 329.0 2.77 ... 0.0 50.0 Starting 1 wax 6 4.0 unworn yes yes
1 198.0 -10.8 -350.00 198.0 -13.6 -358.000000 -10.90 0.1860 328.0 23.30 ... 4.0 50.0 Prep 1 wax 6 4.0 unworn yes yes
2 196.0 -17.8 -6.25 196.0 -17.9 -0.000095 -8.59 0.1400 328.0 30.60 ... 7.0 50.0 Prep 1 wax 6 4.0 unworn yes yes

3 rows × 55 columns

Label Normalization

Count of 'Starting' and 'end' label in Machining_Process column is relatevely small.
So we need to normalize these outlier labels into alternative label.

  • Starting -> Prep
  • end -> End
    • In [73]:
    df['Machining_Process'].value_counts().sort_index()
    Out[73]:
    End              2585
Layer 1 Down     2655
Layer 1 Up       4085
Layer 2 Down     2528
Layer 2 Up       3104
Layer 3 Down     2354
Layer 3 Up       2794
Prep             1795
Repositioning    3377
Starting            1
end                 8
Name: Machining_Process, dtype: int64
    In [74]:
    print(f"Count of Starting label in Machining_Process column is 1 only in experimet_{df[df['Machining_Process']=='Starting'].exp_num.value_counts().index[0]}")
print(f"Count of end label in Machining_Process column is 8 only in experimet_{df[df['Machining_Process']=='end'].exp_num.value_counts().index[0]}")

    Count of Starting label in Machining_Process column is 1 only in experimet_1
Count of end label in Machining_Process column is 8 only in experimet_1
    In [75]:
    df.replace({'Machining_Process': {'Starting':'Prep','end':'End'}}, inplace=True)

    5. EDA

    Some points to focus on

    • Machining Process Unique
    • Mean value of velocity,voltage,feedrate(x,y,z)
    • distribution of feedrate, clamp_pressure
    • correlation of each features
    • time series plot of each experiment
    • difference of distribution for each output feature

    Basic Analysis

    Machine Inputs

    Observation of material is only 'wax'. So this column is not applicable to modeling feature.

    In [76]:
    feedrate = hv.Distribution(df['feedrate']).opts(title="Distribution of feedrate", color="green", xlabel="Feedrate", ylabel="Density")
clamp = hv.Distribution(df['clamp_pressure']).opts(title="Distribution of clamp pressure", color="green", xlabel="Pressure", ylabel="Density")
(feedrate + clamp ).opts(opts.Bars(width=300, height=300,tools=['hover'],show_grid=True)).cols(2)
    Out[76]:

    Machine Outputs

  • Count of 'Not Passed Visual Insepction' is larger than that of 'Not Finalized Machining'.

    This means milling by machineries is not reliable enough, and humans have some tricks or technique to detect failures, which machineries can't do.

  • Count of 'Not Finalized Machining' with worn tool is almost same as that with unworn tool.
  • Count of 'Not Passed Visual Insepction' with worn tool is relatively larger than that with unworn tool.

    It is thought that machining with worn tool does not affect machinig process itself, but humans can detect a slight difference affected by worn tools.

    • In [10]:
    tool_df = np.round(df['tool_condition'].value_counts(normalize=True) * 100)
finalized_df = np.round(df['machining_finalized'].value_counts(normalize=True) * 100)
vis_passed_df = np.round(df['passed_visual_inspection'].value_counts(normalize=True) * 100)
tool_wear = hv.Bars(tool_df).opts(title="Tool Wear Count", color="green", xlabel="Worn/Unworn", ylabel="Percentage", yformatter='%d%%')
finalized = hv.Bars(finalized_df).opts(title="Finalized Count", color="green", xlabel="Yes/No", ylabel="Percentage", yformatter='%d%%')
vis_inspection = hv.Bars(vis_passed_df).opts(title="Visual Inspection Passed Count", color="green", xlabel="Yes/No", ylabel="Percentage", yformatter='%d%%')
(tool_wear + finalized + vis_inspection).opts(opts.Bars(width=300, height=300,tools=['hover'],show_grid=True)).cols(2)
    Out[10]:
    In [11]:
    finalized_df_worn = np.round(df[df['tool_condition']=='worn']['machining_finalized'].value_counts(normalize=True) * 100)
finalized_df_unworn = np.round(df[df['tool_condition']=='unworn']['machining_finalized'].value_counts(normalize=True) * 100)
vis_passed_df_worn = np.round(df[df['tool_condition']=='worn']['passed_visual_inspection'].value_counts(normalize=True) * 100)
vis_passed_df_unworn = np.round(df[df['tool_condition']=='unworn']['passed_visual_inspection'].value_counts(normalize=True) * 100)
finalized_worn = hv.Bars(finalized_df_worn).opts(title="[WORN] Finalized Count", color="orange", xlabel="Yes/No", ylabel="Percentage", yformatter='%d%%')\
            * hv.Text('yes', 15, f"{np.round(finalized_df_worn['yes']/sum(finalized_df_worn)*100)}%")\
            * hv.Text('no', 15, f"{np.round(finalized_df_worn['no']/sum(finalized_df_worn)*100)}%")
finalized_unworn = hv.Bars(finalized_df_unworn).opts(title="[UNWORN] Finalized Count", color="orange", xlabel="Yes/No", ylabel="Percentage", yformatter='%d%%')\
            * hv.Text('yes', 15, f"{np.round(finalized_df_unworn['yes']/sum(finalized_df_unworn)*100)}%")\
            * hv.Text('no', 15, f"{np.round(finalized_df_unworn['no']/sum(finalized_df_unworn)*100)}%")
vis_inspection_worn = hv.Bars(vis_passed_df_worn).opts(title="[WORN] Visual Inspection Passed Count", color="green", xlabel="Yes/No", ylabel="Percentage", yformatter='%d%%')\
            * hv.Text('yes', 45, f"{np.round(vis_passed_df_worn['yes']/sum(vis_passed_df_worn)*100)}%")\
            * hv.Text('no', 45, f"{np.round(vis_passed_df_worn['no']/sum(vis_passed_df_worn)*100)}%")
vis_inspection_unworn = hv.Bars(vis_passed_df_unworn).opts(title="[UNWORN] Visual Inspection Passed Count", color="green", xlabel="Yes/No", ylabel="Percentage", yformatter='%d%%')\
            * hv.Text('yes', 15, f"{np.round(vis_passed_df_unworn['yes']/sum(vis_passed_df_unworn)*100)}%")\
            * hv.Text('no', 15, f"{np.round(vis_passed_df_unworn['no']/sum(vis_passed_df_unworn)*100)}%")
((finalized_worn + finalized_unworn) + (vis_inspection_worn + vis_inspection_unworn)).opts(opts.Bars(width=400, height=300,tools=['hover'],show_grid=True)).cols(2)
    Out[11]:
    In [12]:
    worn_fin_vis = pd.concat([finalized_df_worn, vis_passed_df_worn], axis=1,sort=True).rename(columns={'machining_finalized':'[WORN] Finalized', 'passed_visual_inspection':'[WORN] Visual Inspection Passed'})
worn_fin_vis = pd.melt(worn_fin_vis.reset_index(), ['index']).rename(columns={'index':'Yes/No', 'variable':'Outputs'})
hv.Bars(worn_fin_vis, ['Outputs','Yes/No'], 'value').opts(opts.Bars(title="Machining Finalized and Passed Visual Inspection by Worn Tool Count", width=700, height=400,tools=['hover'],\
                                                                show_grid=True, ylabel="Percentage", yformatter='%d%%'))
    Out[12]:

    Univariate Analysis

    Machining Process

    In [13]:
    hv.Bars(df['Machining_Process'].value_counts()).opts(title="Machining Process Count", color="red", xlabel="Machining Processes", ylabel="Count")\
                                        .opts(opts.Bars(width=500, height=300,tools=['hover'],xrotation=45,show_grid=True))
    Out[13]:

    plot function to output all experiment time-series

    In [14]:
    def plot_ts(col, color='red', yformat='%d%%'):
    v_list = []
    for i in range(1,19):
        v = hv.Curve(df[df['exp_num']==i].reset_index()[col]).opts(title=f"{col} in  experiment {i}", xlabel="Time", ylabel=f"{col}", yformatter=yformat)\
                                                          .opts(width=300, height=150,tools=['hover'],show_grid=True,fontsize=8, color=color)
        v_list.append(v)
    return (v_list[0] + v_list[1] + v_list[2] + v_list[3] + v_list[4] + v_list[5] + v_list[6] + v_list[7] + v_list[8] + v_list[9] + v_list[10] + v_list[11] + v_list[12]\
            + v_list[13] + v_list[14] + v_list[15] + v_list[16] + v_list[17]).opts(shared_axes=False).cols(6)

    Velocity

  • experiments which had worn tools are not distinctive.
  • experiments which did not finalize machining and pass visual inspection are seem to have certain patterns in X/Y/S axes.

    wide range of change
    low frequecy of change

    • In [15]:
    plot_ts('X1_ActualVelocity', color='red', yformat='%d mm/s')
    Out[15]:
    In [16]:
    plot_ts('Y1_ActualVelocity', color='orange', yformat='%d mm/s')
    Out[16]:
    In [17]:
    plot_ts('Z1_ActualVelocity', color='green', yformat='%d mm/s')
    Out[17]:
    In [18]:
    plot_ts('S1_ActualVelocity', color='blue', yformat='%d mm/s')
    Out[18]:

    Current¶

  • experiments which had worn tools are not distinctive.
  • experiments which did not finalize machining and pass visual inspection are seem to have low frequecy of change in X/Y/S axes.
    • In [19]:
    plot_ts('X1_CurrentFeedback', color='red', yformat='%d A')
    Out[19]:
    In [20]:
    plot_ts('Y1_CurrentFeedback', color='orange', yformat='%d A')
    Out[20]:
    In [21]:
    plot_ts('Z1_CurrentFeedback', color='green', yformat='%d A')
    Out[21]:
    In [22]:
    plot_ts('S1_CurrentFeedback', color='blue', yformat='%d A')
    Out[22]:

    Voltage

  • experiments which had worn tools are not distinctive.
  • experiments which did not finalize machining and pass visual inspection are seem to have low frequecy of change in X/Y/S axis.
    • In [23]:
    plot_ts('X1_DCBusVoltage', color='red', yformat='%.1f V')
    Out[23]:
    In [24]:
    plot_ts('Y1_DCBusVoltage', color='orange', yformat='%.1f V')
    Out[24]:
    In [25]:
    plot_ts('Z1_DCBusVoltage', color='green', yformat='%.1f V')
    Out[25]:
    In [26]:
    plot_ts('S1_DCBusVoltage', color='maroon', yformat='%.1f V')
    Out[26]:

    Multivariate Analysis

    Feedrate / Clamp Pressure

    Tool Condition

  • distributions of clamp pressure with unworn/worn tool are almost same.
  • distribution of feedrate with worn tool is more wider than that with unworn tool.

    • Machining Finalized

  • distribution of clamp pressure in machining not finalized experiment has lower tailedness.
  • distribution of feedrate in machining not finalized experiment has higher tailedness.
    • In [27]:
    g = sns.pairplot(df, hue='tool_condition', vars=["feedrate","clamp_pressure"])
g.fig.suptitle("Tool Condition - feedrate/clamp pressure", y=1.1, fontsize=20)
plt.show()
    In [28]:
    g = sns.pairplot(df, hue='machining_finalized', vars=["feedrate","clamp_pressure"])
g.fig.suptitle("Machining Finalized - feedrate/clamp pressure", y=1.1, fontsize=20)
g.fig.set_figheight(6)
g.fig.set_figwidth(9)
plt.show()

    Velocity

    Machining Finalized

  • distribution of velocity in S axis in machining not finalized experiment has lower tailedness.
    • In [29]:
    g = sns.pairplot(df, hue='tool_condition', vars=['X1_ActualVelocity','Y1_ActualVelocity','Z1_ActualVelocity','S1_ActualVelocity'])
g.fig.suptitle("Tool Condition - velocity", y=1.1, fontsize=20)
g.fig.set_figheight(6)
g.fig.set_figwidth(9)
g.fig.get_children()[-1].set_bbox_to_anchor((1.1, 0.5, 0, 0))
plt.show()
    In [30]:
    g = sns.pairplot(df, hue='machining_finalized', vars=['X1_ActualVelocity','Y1_ActualVelocity','Z1_ActualVelocity','S1_ActualVelocity'])
g.fig.suptitle("Machining Finalized - velocity", y=1.1, fontsize=20)
g.fig.set_figheight(6)
g.fig.set_figwidth(9)
g.fig.get_children()[-1].set_bbox_to_anchor((1.1, 0.5, 0, 0))
plt.show()

    Current

    Machining Finalized

  • distribution of current in X/Y/S axes in machining not finalized experiment has higher tailedness.
    • In [31]:
    np.seterr(divide='ignore', invalid='ignore')
g = sns.pairplot(df, hue='tool_condition', vars=['X1_CurrentFeedback','Y1_CurrentFeedback','Z1_CurrentFeedback','S1_CurrentFeedback'])
g.fig.suptitle("Tool Condition - Current", y=1.1, fontsize=20)
g.fig.set_figheight(6)
g.fig.set_figwidth(9)
g.fig.get_children()[-1].set_bbox_to_anchor((1.1, 0.5, 0, 0))
plt.show()
    In [32]:
    g = sns.pairplot(df, hue='machining_finalized', vars=['X1_CurrentFeedback','Y1_CurrentFeedback','Z1_CurrentFeedback','S1_CurrentFeedback'])
g.fig.suptitle("Machining Finalized - Current", y=1.1, fontsize=20)
g.fig.set_figheight(6)
g.fig.set_figwidth(9)
g.fig.get_children()[-1].set_bbox_to_anchor((1.1, 0.5, 0, 0))
plt.show()

    Voltage

    Machining Finalized

    • distribution of voltage in X/Y/S axis in machining not finalized experiment has higher skewness.
    In [33]:
    g = sns.pairplot(df, hue='tool_condition', vars=['X1_DCBusVoltage','Y1_DCBusVoltage','Z1_DCBusVoltage','S1_DCBusVoltage'])
g.fig.suptitle("Tool Condition - Voltage", y=1.1, fontsize=20)
g.fig.set_figheight(6)
g.fig.set_figwidth(9)
g.fig.get_children()[-1].set_bbox_to_anchor((1.1, 0.5, 0, 0))
plt.show()
    In [34]:
    g = sns.pairplot(df, hue='machining_finalized', vars=['X1_DCBusVoltage','Y1_DCBusVoltage','Z1_DCBusVoltage','S1_DCBusVoltage'])
g.fig.suptitle("Machining Finalized - Voltage", y=1.1, fontsize=20)
g.fig.set_figheight(6)
g.fig.set_figwidth(9)
g.fig.get_children()[-1].set_bbox_to_anchor((1.1, 0.5, 0, 0))
plt.show()

    Frequency Analysis

    • experiments which did not finalize machining seem to have high amplitude in certain frequencies.
    • experiments which did not pass visual inspection seem to have high amplitude in certain frequencies too.

      this is not so obvious in univariate plot above.
      these discoveries observed in frequency analysis can be related to why humans can detect machining failures.

    plot function to output all experiment fft result

    In [35]:
    def plot_fft(col, color='red', peak_thr=1):
    v_list = []
    dt = 0.1 # experiment data was collected per 100ms(0.1sec)
    for i in range(1,19):
        f = df[df['exp_num']==i].reset_index()[col]
        N = len(f)
        t = np.arange(0, N*dt, dt)
        freq = np.linspace(0, 1.0/dt, N)
        F = np.fft.fft(f)
        F_abs = np.abs(F) / (N/2) 
        F_abs[0] = F_abs[0] / 2
        
        maximal_idx = signal.argrelmax(F_abs, order=1)[0] 
        peak_cut = peak_thr
        maximal_idx = maximal_idx[(F_abs[maximal_idx] > peak_cut) & (maximal_idx <= N/2)]
        
        v = hv.Curve((freq[:int(N/2)+1], F_abs[:int(N/2)+1])).opts(title=f"{col} in  experiment {i}", xlabel="Frequency(Hz)", ylabel=f"Amplitude")\
                                                          .opts(width=300, height=150,tools=['hover'],show_grid=True,fontsize=8, color=color)\
            * hv.Scatter((freq[maximal_idx], F_abs[maximal_idx])).opts(color='lime', size=5)
        
        v_list.append(v)
    return (v_list[0] + v_list[1] + v_list[2] + v_list[3] + v_list[4] + v_list[5] + v_list[6] + v_list[7] + v_list[8] + v_list[9] + v_list[10] + v_list[11] + v_list[12]\
            + v_list[13] + v_list[14] + v_list[15] + v_list[16] + v_list[17]).opts(shared_axes=False).cols(6)

    Velocity

    In [36]:
    plot_fft('X1_ActualVelocity', color='red', peak_thr=3)
    Out[36]:
    In [37]:
    plot_fft('Y1_ActualVelocity', color='orange', peak_thr=3)
    Out[37]:
    In [38]:
    plot_fft('Z1_ActualVelocity', color='green', peak_thr=3)
    Out[38]:
    In [39]:
    plot_fft('S1_ActualVelocity', color='blue', peak_thr=9)
    Out[39]:

    Current

    In [40]:
    plot_fft('X1_CurrentFeedback', color='red', peak_thr=1.2)
    Out[40]:
    In [41]:
    plot_fft('Y1_CurrentFeedback', color='orange', peak_thr=1.2)
    Out[41]:
    In [42]:
    plot_fft('Z1_CurrentFeedback', color='green', peak_thr=3)
    Out[42]:
    In [43]:
    plot_fft('X1_CurrentFeedback', color='blue', peak_thr=1.2)
    Out[43]:

    Volatage

    In [44]:
    plot_fft('X1_DCBusVoltage', color='red', peak_thr=0.015)
    Out[44]:
    In [45]:
    plot_fft('Y1_DCBusVoltage', color='orange', peak_thr=0.02)
    Out[45]:
    In [46]:
    plot_fft('Z1_DCBusVoltage', color='green', peak_thr=3)
    Out[46]:
    In [47]:
    plot_fft('S1_DCBusVoltage', color='blue', peak_thr=0.15)
    Out[47]:

    6. Modeling

    • Target variables :
      • tool_condition
      • machining_finalized
      • passed_visual_inspection
    • Through some experiment cases, I examined which features are more important to make robust prediction models.
      • Especially I wanted to know which CNC imformation, for example tool position or current, is most important to detect machine failure.

    Feature Engineering

    Differential Features

  • differences between Actual and Command position can indicate a sign of maching failure.
    • In [48]:
    for ax in ['X','Y','Z','S']:
    df[f'{ax}1_Position_Diff'] = abs(df[f'{ax}1_CommandPosition']-df[f'{ax}1_ActualPosition'])
    df[f'{ax}1_Velocity_Diff'] = abs(df[f'{ax}1_CommandVelocity']-df[f'{ax}1_ActualVelocity'])
    df[f'{ax}1_Acceleration_Diff'] = abs(df[f'{ax}1_CommandAcceleration']-df[f'{ax}1_ActualAcceleration'])

    FFT Features

  • it is thought that highest frequency and amplitude can be good information to detect machinig failures.
    • In [49]:
    for col in ['ActualPosition','ActualVelocity','ActualAcceleration','CurrentFeedback','DCBusVoltage','OutputCurrent','OutputVoltage','OutputPower']:
    dt = 0.1
    for i in range(1,19):
        for ax in ['X','Y','Z','S']:
            try:
                f = df[df['exp_num']==i].reset_index()[f'{ax}1_{col}']
            except:
                continue
                
            N = len(f)
            t = np.arange(0, N*dt, dt)
            freq = np.linspace(0, 1.0/dt, N)
            F = np.fft.fft(f)
            F_abs = np.abs(F) / (N/2) 
            F_abs[0] = F_abs[0] / 2
            maximal_idx = signal.argrelmax(F_abs, order=1)[0]

            high_amp = np.max(F_abs[maximal_idx]) if len(maximal_idx) > 0 else 0
            high_freq = freq[maximal_idx][np.argmax(F_abs[maximal_idx])] if len(maximal_idx) > 0 else 0

            df.loc[df['exp_num']==i,f'{ax}1_{col}_High_Amp'] = high_amp
            df.loc[df['exp_num']==i,f'{ax}1_{col}_High_Freq'] = high_freq
            df.loc[df['exp_num']==i,f'{ax}1_{col}_High_Amp_Freq'] = high_amp * high_freq

    using label encoder to drop unnecessary columns in the Dataframe¶

    In [50]:
    feature_df = df.copy()
feature_df['Machining_Process'] = LabelEncoder().fit_transform(feature_df['Machining_Process']).astype(np.int8)
feature_df['tool_condition'] = LabelEncoder().fit_transform(feature_df['tool_condition']).astype(np.int8)
feature_df['machining_finalized'] = LabelEncoder().fit_transform(feature_df['machining_finalized']).astype(np.int8)
feature_df['passed_visual_inspection'] = LabelEncoder().fit_transform(feature_df['passed_visual_inspection']).astype(np.int8)
feature_df.drop(['material','exp_num'], axis=1, inplace=True)
feature_df.head(3)
    Out[50]:
    X1_ActualPosition X1_ActualVelocity X1_ActualAcceleration X1_CommandPosition X1_CommandVelocity X1_CommandAcceleration X1_CurrentFeedback X1_DCBusVoltage X1_OutputCurrent X1_OutputVoltage ... S1_OutputVoltage_High_Amp_Freq X1_OutputPower_High_Amp X1_OutputPower_High_Freq X1_OutputPower_High_Amp_Freq Y1_OutputPower_High_Amp Y1_OutputPower_High_Freq Y1_OutputPower_High_Amp_Freq S1_OutputPower_High_Amp S1_OutputPower_High_Freq S1_OutputPower_High_Amp_Freq
    0 198.0 0.0 0.00 198.0 0.0 0.000000 0.18 0.0207 329.0 2.77 ... 0.222398 0.000492 0.199241 0.000098 0.000725 9.867173 0.00715 0.011293 3.358634 0.03793
    1 198.0 -10.8 -350.00 198.0 -13.6 -358.000000 -10.90 0.1860 328.0 23.30 ... 0.222398 0.000492 0.199241 0.000098 0.000725 9.867173 0.00715 0.011293 3.358634 0.03793
    2 196.0 -17.8 -6.25 196.0 -17.9 -0.000095 -8.59 0.1400 328.0 30.60 ... 0.222398 0.000492 0.199241 0.000098 0.000725 9.867173 0.00715 0.011293 3.358634 0.03793

    3 rows × 158 columns

    Case1 : Tool Condition

    In [51]:
    y_series = feature_df['tool_condition']
x_df = feature_df.drop(['tool_condition','machining_finalized','passed_visual_inspection'], axis=1) 
X_train, X_valid, Y_train, Y_valid = train_test_split(x_df, y_series, test_size=0.2, random_state=0, stratify=y_series)

lgb_train = lgb.Dataset(X_train, Y_train)
lgb_valid = lgb.Dataset(X_valid, Y_valid, reference=lgb_train)
    In [52]:
    params = {
    'task' : 'train',
    'boosting' : 'gbdt',
    'objective': 'binary',
    'metric': 'l2',
    'num_leaves': 200,
    'feature_fraction': 1.0,
    'bagging_fraction': 1.0,
    'bagging_freq': 0,
    'min_child_samples': 5
}
gbm_tool_wear = lgb.train(params,
            lgb_train,
            num_boost_round=100,
            valid_sets=lgb_valid,
            early_stopping_rounds=100)

    [LightGBM] [Info] Number of positive: 10646, number of negative: 9582
[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.006454 seconds.
You can set `force_col_wise=true` to remove the overhead.
[LightGBM] [Info] Total Bins 10481
[LightGBM] [Info] Number of data points in the train set: 20228, number of used features: 138
[LightGBM] [Info] [binary:BoostFromScore]: pavg=0.526300 -> initscore=0.105298
[LightGBM] [Info] Start training from score 0.105298
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[1]	valid_0's l2: 0.202065
Training until validation scores don't improve for 100 rounds
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[2]	valid_0's l2: 0.164086
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[3]	valid_0's l2: 0.133432
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[4]	valid_0's l2: 0.108622
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[5]	valid_0's l2: 0.0885
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[6]	valid_0's l2: 0.0721553
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[7]	valid_0's l2: 0.0588624
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[8]	valid_0's l2: 0.0480409
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[9]	valid_0's l2: 0.0392243
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[10]	valid_0's l2: 0.0320365
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[11]	valid_0's l2: 0.0261733
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[12]	valid_0's l2: 0.0213885
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[13]	valid_0's l2: 0.0174821
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[14]	valid_0's l2: 0.0142918
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[15]	valid_0's l2: 0.0116857
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[16]	valid_0's l2: 0.00955611
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[17]	valid_0's l2: 0.00781562
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[18]	valid_0's l2: 0.00639284
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[19]	valid_0's l2: 0.00522958
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[20]	valid_0's l2: 0.00427837
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[21]	valid_0's l2: 0.00350045
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[22]	valid_0's l2: 0.00286417
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[23]	valid_0's l2: 0.00234369
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[24]	valid_0's l2: 0.00191791
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[25]	valid_0's l2: 0.00156955
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[26]	valid_0's l2: 0.00128452
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[27]	valid_0's l2: 0.0010513
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[28]	valid_0's l2: 0.000860449
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[29]	valid_0's l2: 0.000704269
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[30]	valid_0's l2: 0.000576454
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[31]	valid_0's l2: 0.000471848
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[32]	valid_0's l2: 0.000386233
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[33]	valid_0's l2: 0.000316159
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[34]	valid_0's l2: 0.000258804
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[35]	valid_0's l2: 0.000211857
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[36]	valid_0's l2: 0.000173429
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[37]	valid_0's l2: 0.000141973
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[38]	valid_0's l2: 0.000116224
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[39]	valid_0's l2: 9.51463e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[40]	valid_0's l2: 7.78917e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[41]	valid_0's l2: 6.37668e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[42]	valid_0's l2: 5.22038e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[43]	valid_0's l2: 4.27378e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[44]	valid_0's l2: 3.49885e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[45]	valid_0's l2: 2.86445e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[46]	valid_0's l2: 2.34509e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[47]	valid_0's l2: 1.91991e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[48]	valid_0's l2: 1.57182e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[49]	valid_0's l2: 1.28685e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[50]	valid_0's l2: 1.05355e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[51]	valid_0's l2: 8.62543e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[52]	valid_0's l2: 7.0617e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[53]	valid_0's l2: 5.78148e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[54]	valid_0's l2: 4.73337e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[55]	valid_0's l2: 3.87527e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[56]	valid_0's l2: 3.17274e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[57]	valid_0's l2: 2.59758e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[58]	valid_0's l2: 2.12668e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[59]	valid_0's l2: 1.74116e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[60]	valid_0's l2: 1.42552e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[61]	valid_0's l2: 1.1671e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[62]	valid_0's l2: 9.55534e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[63]	valid_0's l2: 7.82317e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[64]	valid_0's l2: 6.40502e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[65]	valid_0's l2: 5.24394e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[66]	valid_0's l2: 4.29335e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[67]	valid_0's l2: 3.51507e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[68]	valid_0's l2: 2.87788e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[69]	valid_0's l2: 2.3562e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[70]	valid_0's l2: 1.92908e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[71]	valid_0's l2: 1.57939e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[72]	valid_0's l2: 1.29309e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[73]	valid_0's l2: 1.05869e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[74]	valid_0's l2: 8.6678e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[75]	valid_0's l2: 7.09657e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[76]	valid_0's l2: 5.81017e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[77]	valid_0's l2: 4.75695e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[78]	valid_0's l2: 3.89466e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[79]	valid_0's l2: 3.18867e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[80]	valid_0's l2: 2.61066e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[81]	valid_0's l2: 2.13742e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[82]	valid_0's l2: 1.74997e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[83]	valid_0's l2: 1.43275e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[84]	valid_0's l2: 1.17304e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[85]	valid_0's l2: 9.604e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[86]	valid_0's l2: 7.86309e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[87]	valid_0's l2: 6.43774e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[88]	valid_0's l2: 5.27078e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[89]	valid_0's l2: 4.31534e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[90]	valid_0's l2: 3.5331e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[91]	valid_0's l2: 2.89266e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[92]	valid_0's l2: 2.36831e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[93]	valid_0's l2: 1.939e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[94]	valid_0's l2: 1.58752e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[95]	valid_0's l2: 1.29975e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[96]	valid_0's l2: 1.06415e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[97]	valid_0's l2: 8.7125e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[98]	valid_0's l2: 7.13319e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[99]	valid_0's l2: 5.84016e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[100]	valid_0's l2: 4.78152e-10
Did not meet early stopping. Best iteration is:
[100]	valid_0's l2: 4.78152e-10
    In [53]:
    feature_imp = pd.DataFrame()
feature_imp['feature'] = gbm_tool_wear.feature_name()
feature_imp['importance'] = gbm_tool_wear.feature_importance()
hv.Bars(feature_imp.sort_values(by='importance', ascending=False)[0:31][::-1]).opts(title="Feature Importance", color="purple", xlabel="Features", ylabel="Importance", invert_axes=True)\
                            .opts(opts.Bars(width=700, height=700, tools=['hover'], show_grid=True))
    Out[53]:

    Case 2 : machine finalized

    In [54]:
    y_series = feature_df['machining_finalized']
x_df = feature_df.drop(['tool_condition','machining_finalized','passed_visual_inspection'], axis=1) 
X_train, X_valid, Y_train, Y_valid = train_test_split(x_df, y_series, test_size=0.2, random_state=0, stratify=y_series)

lgb_train = lgb.Dataset(X_train, Y_train)
lgb_valid = lgb.Dataset(X_valid, Y_valid, reference=lgb_train)
    In [55]:
    params = {
    'task' : 'train',
    'boosting' : 'gbdt',
    'objective': 'binary',
    'metric': 'l2',
    'num_leaves': 200,
    'feature_fraction': 1.0,
    'bagging_fraction': 1.0,
    'bagging_freq': 0,
    'min_child_samples': 5
}
gbm_machining_finalized = lgb.train(params,
            lgb_train,
            num_boost_round=100,
            valid_sets=lgb_valid,
            early_stopping_rounds=100)

    [LightGBM] [Info] Number of positive: 18499, number of negative: 1729
[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.006445 seconds.
You can set `force_col_wise=true` to remove the overhead.
[LightGBM] [Info] Total Bins 10505
[LightGBM] [Info] Number of data points in the train set: 20228, number of used features: 138
[LightGBM] [Info] [binary:BoostFromScore]: pavg=0.914524 -> initscore=2.370173
[LightGBM] [Info] Start training from score 2.370173
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[1]	valid_0's l2: 0.055917
Training until validation scores don't improve for 100 rounds
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[2]	valid_0's l2: 0.0443281
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[3]	valid_0's l2: 0.0355696
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[4]	valid_0's l2: 0.0287056
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[5]	valid_0's l2: 0.023245
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[6]	valid_0's l2: 0.0188657
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[7]	valid_0's l2: 0.0153361
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[8]	valid_0's l2: 0.0124818
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[9]	valid_0's l2: 0.0101682
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[10]	valid_0's l2: 0.00828952
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[11]	valid_0's l2: 0.00676199
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[12]	valid_0's l2: 0.00551865
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[13]	valid_0's l2: 0.00450576
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[14]	valid_0's l2: 0.00368005
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[15]	valid_0's l2: 0.00300654
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[16]	valid_0's l2: 0.00245691
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[17]	valid_0's l2: 0.0020082
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[18]	valid_0's l2: 0.00164174
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[19]	valid_0's l2: 0.00134238
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[20]	valid_0's l2: 0.00109776
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[21]	valid_0's l2: 0.000897827
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[22]	valid_0's l2: 0.000734391
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[23]	valid_0's l2: 0.000600766
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[24]	valid_0's l2: 0.000491497
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[25]	valid_0's l2: 0.000402133
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[26]	valid_0's l2: 0.00032904
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[27]	valid_0's l2: 0.000269249
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[28]	valid_0's l2: 0.000220335
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[29]	valid_0's l2: 0.000180316
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[30]	valid_0's l2: 0.000147572
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[31]	valid_0's l2: 0.000120779
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[32]	valid_0's l2: 9.88535e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[33]	valid_0's l2: 8.0911e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[34]	valid_0's l2: 6.62271e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[35]	valid_0's l2: 5.42094e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[36]	valid_0's l2: 4.43735e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[37]	valid_0's l2: 3.6323e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[38]	valid_0's l2: 2.97336e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[39]	valid_0's l2: 2.434e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[40]	valid_0's l2: 1.99251e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[41]	valid_0's l2: 1.63112e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[42]	valid_0's l2: 1.3353e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[43]	valid_0's l2: 1.09314e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[44]	valid_0's l2: 8.94899e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[45]	valid_0's l2: 7.32619e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[46]	valid_0's l2: 5.99772e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[47]	valid_0's l2: 4.91018e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[48]	valid_0's l2: 4.01986e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[49]	valid_0's l2: 3.291e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[50]	valid_0's l2: 2.6943e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[51]	valid_0's l2: 2.20581e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[52]	valid_0's l2: 1.80589e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[53]	valid_0's l2: 1.47848e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[54]	valid_0's l2: 1.21043e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[55]	valid_0's l2: 9.90989e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[56]	valid_0's l2: 8.1133e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[57]	valid_0's l2: 6.64244e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[58]	valid_0's l2: 5.43825e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[59]	valid_0's l2: 4.45237e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[60]	valid_0's l2: 3.64522e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[61]	valid_0's l2: 2.9844e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[62]	valid_0's l2: 2.44339e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[63]	valid_0's l2: 2.00045e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[64]	valid_0's l2: 1.63781e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[65]	valid_0's l2: 1.34091e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[66]	valid_0's l2: 1.09783e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[67]	valid_0's l2: 8.9882e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[68]	valid_0's l2: 7.35885e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[69]	valid_0's l2: 6.02487e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[70]	valid_0's l2: 4.93271e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[71]	valid_0's l2: 4.03854e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[72]	valid_0's l2: 3.30646e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[73]	valid_0's l2: 2.70708e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[74]	valid_0's l2: 2.21636e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[75]	valid_0's l2: 1.8146e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[76]	valid_0's l2: 1.48566e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[77]	valid_0's l2: 1.21635e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[78]	valid_0's l2: 9.95862e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[79]	valid_0's l2: 8.1534e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[80]	valid_0's l2: 6.67543e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[81]	valid_0's l2: 5.46536e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[82]	valid_0's l2: 4.47465e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[83]	valid_0's l2: 3.66353e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[84]	valid_0's l2: 2.99944e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[85]	valid_0's l2: 2.45573e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[86]	valid_0's l2: 2.01058e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[87]	valid_0's l2: 1.64612e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[88]	valid_0's l2: 1.34773e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[89]	valid_0's l2: 1.10342e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[90]	valid_0's l2: 9.03407e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[91]	valid_0's l2: 7.39646e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[92]	valid_0's l2: 6.05571e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[93]	valid_0's l2: 4.95799e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[94]	valid_0's l2: 4.05926e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[95]	valid_0's l2: 3.32344e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[96]	valid_0's l2: 2.721e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[97]	valid_0's l2: 2.22776e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[98]	valid_0's l2: 1.82394e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[99]	valid_0's l2: 1.49331e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[100]	valid_0's l2: 1.22262e-10
Did not meet early stopping. Best iteration is:
[100]	valid_0's l2: 1.22262e-10
    In [56]:
    feature_imp = pd.DataFrame()
feature_imp['feature'] = gbm_machining_finalized.feature_name()
feature_imp['importance'] = gbm_machining_finalized.feature_importance()
hv.Bars(feature_imp.sort_values(by='importance', ascending=False)[0:31][::-1]).opts(title="Feature Importance", color="purple", xlabel="Features", ylabel="Importance", invert_axes=True)\
                            .opts(opts.Bars(width=700, height=700, tools=['hover'], show_grid=True))
    Out[56]:

    Case3 : Passed Visual Inspection

    In [57]:
    y_series = feature_df['passed_visual_inspection']
x_df = feature_df.drop(['tool_condition','machining_finalized','passed_visual_inspection'], axis=1) 
X_train, X_valid, Y_train, Y_valid = train_test_split(x_df, y_series, test_size=0.2, random_state=0, stratify=y_series)

lgb_train = lgb.Dataset(X_train, Y_train)
lgb_valid = lgb.Dataset(X_valid, Y_valid, reference=lgb_train)
    In [58]:
    params = {
    'task' : 'train',
    'boosting' : 'gbdt',
    'objective': 'binary',
    'metric': 'l2',
    'num_leaves': 200,
    'feature_fraction': 1.0,
    'bagging_fraction': 1.0,
    'bagging_freq': 0,
    'min_child_samples': 5
}
gbm_passed_vis_inspection = lgb.train(params,
            lgb_train,
            num_boost_round=100,
            valid_sets=lgb_valid,
            early_stopping_rounds=100)

    [LightGBM] [Info] Number of positive: 15346, number of negative: 4882
[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.006351 seconds.
You can set `force_col_wise=true` to remove the overhead.
[LightGBM] [Info] Total Bins 10484
[LightGBM] [Info] Number of data points in the train set: 20228, number of used features: 138
[LightGBM] [Info] [binary:BoostFromScore]: pavg=0.758651 -> initscore=1.145300
[LightGBM] [Info] Start training from score 1.145300
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[1]	valid_0's l2: 0.146059
Training until validation scores don't improve for 100 rounds
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[2]	valid_0's l2: 0.117591
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[3]	valid_0's l2: 0.0951075
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[4]	valid_0's l2: 0.0771371
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[5]	valid_0's l2: 0.0626795
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[6]	valid_0's l2: 0.0510003
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[7]	valid_0's l2: 0.0415395
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[8]	valid_0's l2: 0.0338605
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[9]	valid_0's l2: 0.0276185
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[10]	valid_0's l2: 0.0225388
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[11]	valid_0's l2: 0.0184012
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[12]	valid_0's l2: 0.0150285
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[13]	valid_0's l2: 0.0122777
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[14]	valid_0's l2: 0.010033
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[15]	valid_0's l2: 0.0082005
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[16]	valid_0's l2: 0.00670398
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[17]	valid_0's l2: 0.00548148
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[18]	valid_0's l2: 0.00448256
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[19]	valid_0's l2: 0.00366615
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[20]	valid_0's l2: 0.00299876
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[21]	valid_0's l2: 0.00245312
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[22]	valid_0's l2: 0.00200693
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[23]	valid_0's l2: 0.00164202
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[24]	valid_0's l2: 0.00134356
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[25]	valid_0's l2: 0.00109942
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[26]	valid_0's l2: 0.000899685
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[27]	valid_0's l2: 0.000736275
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[28]	valid_0's l2: 0.000602572
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[29]	valid_0's l2: 0.000493169
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[30]	valid_0's l2: 0.000403642
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[31]	valid_0's l2: 0.000330379
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[32]	valid_0's l2: 0.000270421
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[33]	valid_0's l2: 0.000221349
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[34]	valid_0's l2: 0.000181187
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[35]	valid_0's l2: 0.000148315
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[36]	valid_0's l2: 0.000121409
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[37]	valid_0's l2: 9.93857e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[38]	valid_0's l2: 8.13587e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[39]	valid_0's l2: 6.66023e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[40]	valid_0's l2: 5.45231e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[41]	valid_0's l2: 4.46351e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[42]	valid_0's l2: 3.65407e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[43]	valid_0's l2: 2.99144e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[44]	valid_0's l2: 2.449e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[45]	valid_0's l2: 2.00493e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[46]	valid_0's l2: 1.6414e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[47]	valid_0's l2: 1.34378e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[48]	valid_0's l2: 1.10014e-05
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[49]	valid_0's l2: 9.00678e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[50]	valid_0's l2: 7.37381e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[51]	valid_0's l2: 6.03694e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[52]	valid_0's l2: 4.94246e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[53]	valid_0's l2: 4.04642e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[54]	valid_0's l2: 3.31283e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[55]	valid_0's l2: 2.71225e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[56]	valid_0's l2: 2.22055e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[57]	valid_0's l2: 1.818e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[58]	valid_0's l2: 1.48842e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[59]	valid_0's l2: 1.21859e-06
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[60]	valid_0's l2: 9.97686e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[61]	valid_0's l2: 8.16825e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[62]	valid_0's l2: 6.68751e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[63]	valid_0's l2: 5.47521e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[64]	valid_0's l2: 4.48267e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[65]	valid_0's l2: 3.67007e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[66]	valid_0's l2: 3.00477e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[67]	valid_0's l2: 2.46008e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[68]	valid_0's l2: 2.01413e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[69]	valid_0's l2: 1.64902e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[70]	valid_0's l2: 1.3501e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[71]	valid_0's l2: 1.10536e-07
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[72]	valid_0's l2: 9.04988e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[73]	valid_0's l2: 7.40938e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[74]	valid_0's l2: 6.06626e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[75]	valid_0's l2: 4.96662e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[76]	valid_0's l2: 4.06631e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[77]	valid_0's l2: 3.32921e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[78]	valid_0's l2: 2.72572e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[79]	valid_0's l2: 2.23162e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[80]	valid_0's l2: 1.82709e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[81]	valid_0's l2: 1.4959e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[82]	valid_0's l2: 1.22473e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[83]	valid_0's l2: 1.00273e-08
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[84]	valid_0's l2: 8.20961e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[85]	valid_0's l2: 6.72145e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[86]	valid_0's l2: 5.50305e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[87]	valid_0's l2: 4.50551e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[88]	valid_0's l2: 3.6888e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[89]	valid_0's l2: 3.02013e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[90]	valid_0's l2: 2.47267e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[91]	valid_0's l2: 2.02445e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[92]	valid_0's l2: 1.65748e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[93]	valid_0's l2: 1.35703e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[94]	valid_0's l2: 1.11104e-09
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[95]	valid_0's l2: 9.09643e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[96]	valid_0's l2: 7.44752e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[97]	valid_0's l2: 6.09751e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[98]	valid_0's l2: 4.99222e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[99]	valid_0's l2: 4.08728e-10
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[100]	valid_0's l2: 3.34638e-10
Did not meet early stopping. Best iteration is:
[100]	valid_0's l2: 3.34638e-10
    In [59]:
    feature_imp = pd.DataFrame()
feature_imp['feature'] = gbm_passed_vis_inspection.feature_name()
feature_imp['importance'] = gbm_passed_vis_inspection.feature_importance()
hv.Bars(feature_imp.sort_values(by='importance', ascending=False)[0:31][::-1]).opts(title="Feature Importance", color="purple", xlabel="Features", ylabel="Importance", invert_axes=True)\
                            .opts(opts.Bars(width=700, height=700, tools=['hover'], show_grid=True))
    Out[59]:

    Conclusion

    Machining Finalized

    • X-axis and Y-axis data have a great influence to tool wears, and the movement of X-axis and Y-axis can have a bad effect to machining process.
    • Position and voltage have a large influence in the given data.

    Tool Wear

    • X-axis and S-axis data have a great influence to tool wear.
    • Velocity and voltage have a large influence in the collected data.
    • A certain number of features created by FFT are also included in the top of feature importances, and it is considered that tool wear increased the amplitude in the specific frequency range.

    Passed Visual Inspection

    • X-axis and Y-axis data have a great influence to tool wears, and the movement of X-axis and Y-axis may be a bad affect to machining process.
    • Velocity and voltage have a large influence in the collected data.
    • The importance of features created by FFT increased compared to Machining Finalized case. 
        <p>It is considered that the influence of tool weariness to the specific frequency range is <u>related to an external processing result that can be detected by Machine operator</u>.</p></li>

    Others

    • There is almost less importance in Z-axis data.
    • The differential features do not have a large influence on the detection of anomalies throughout, and it is thought that the difference between the command and the actual position was negligibly small for tool wear and machining process in most cases.

    In order to proceed to more detailed analysis, it is necessary to deepen the understanding of the operating principles of processing machines.